<html>
<head><meta charset="utf-8"><title>Muckable Pitfall - Partial Visibility · project-safe-transmute · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/index.html">project-safe-transmute</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html">Muckable Pitfall - Partial Visibility</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="218478354"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218478354" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218478354">(Dec 01 2020 at 20:40)</a>:</h4>
<p>For a well-defined transmutation to be safe, all fields which might be either instantiated or mutated by the transmutation must be visible in the scope the transmutation occurs. (Private fields might be subject to library invariants, and thus cannot be fiddled with arbitrarily.)</p>
<p>If a module defines a type:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[repr(C)]</span><span class="w"></span>
<span class="k">struct</span> <span class="nc">Packet</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">  </span><span class="k">pub</span><span class="w"> </span><span class="n">header</span>: <span class="nc">Header</span><span class="p">,</span><span class="w"></span>
<span class="w">  </span><span class="n">body</span>: <span class="nc">Body</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>...it is perfectly safe for consumers of that module to transmute a <code>&amp;mut Packet</code> into a <code>&amp;mut Header</code> (or anything that <code>Header</code> is transmutable into; e.g., perhaps an array of bytes). The foundational API can confirm that this is safe, because it is aware of the visibility of fields.</p>
<p>A <code>Muckable</code>-style approach cannot reason about the visibility of fields. Implementing <code>Muckable</code> for a type essentially overrides the visibility of that type's fields with the visibility of the type itself. <a href="#narrow/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Encapsulation.20Redundancy.20Risks/near/218330313">See here for more details.</a></p>



<a name="218485789"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218485789" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218485789">(Dec 01 2020 at 21:40)</a>:</h4>
<p>IMO <code>Scope</code> does not actually help here. It simply trades one imprecision (<code>Muckable</code> exposes the entire layout) for another (everything about the layout of public fields). E.g. today <code>Packet</code> could arguably reorder its fields; the type author did not necessarily want to expose the <em>position</em> of the <code>header</code> field just because it is <code>pub</code>. The archetype pattern seems more useful here- the type author could expose transmutation through a <code>Muckable</code> proxy type with "nothing" in the place of its private fields to denote a public field offset.</p>



<a name="218489096"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218489096" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218489096">(Dec 01 2020 at 22:08)</a>:</h4>
<p>Yes, <code>Packet</code> must still document its layout stability guarantees. The foundational API doesn't say anything about semver stability.</p>



<a name="218489281"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218489281" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218489281">(Dec 01 2020 at 22:10)</a>:</h4>
<p>I used Packet as an example because its layout is meant to mirror a wire format. So, reordering fields isn't viable in this case.</p>



<a name="218489671"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218489671" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218489671">(Dec 01 2020 at 22:12)</a>:</h4>
<p>Stability notwithstanding, the foundational API is able to tell you that this transmutation is safe—without you redesigning the type.</p>



<a name="218489984"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218489984" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218489984">(Dec 01 2020 at 22:15)</a>:</h4>
<p>I'd love to see another attempt at an archetype based stability system, but the last one wasn't sound and <code>Muckable</code> seemed more attractive to RFC commenters.</p>



<a name="218490367"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218490367" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218490367">(Dec 01 2020 at 22:18)</a>:</h4>
<p>Anyways, visibility is more nuanced than pub and pub(self). The foundational API can reason about this. In contrast, <a href="#narrow/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Encapsulation.20Redundancy.20Risks/near/218330313">any trait based system that serves as a proxy for field visibility is going to clobber field visibility in favor of type visibility.</a> I don't see how the archetype system would resolve this.</p>



<a name="218505623"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505623" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505623">(Dec 02 2020 at 01:34)</a>:</h4>
<p>If the foundational API doesn't take semver into account then why does it need to take visibility into account either? Visibility is how you define semver...</p>



<a name="218505770"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505770" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505770">(Dec 02 2020 at 01:36)</a>:</h4>
<p>Visibility <em>isn't</em> how you define layout semver guarantees. Layout semver guarantees are totally new ground in Rust.</p>



<a name="218505806"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505806" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505806">(Dec 02 2020 at 01:37)</a>:</h4>
<p>Okay, but visibility <em>is</em> how you define semver guarantees for things that do exist. Even things like <code>#[non_exhaustive]</code> that aren't attached to the <code>pub</code> keyword are visibility- they restrict what details external code can exploit.</p>



<a name="218505882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505882">(Dec 02 2020 at 01:38)</a>:</h4>
<p>And for that matter, an archetype system "clobbering field visibility in favor of type visibility" seems like a fine way to tie layout to visibility- the archetype system is based on a new type with its own visibility, that's the whole point.</p>



<a name="218505891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505891">(Dec 02 2020 at 01:38)</a>:</h4>
<p>In practice unsafe code has been defining layout guarantees for a while. For example <a href="https://docs.rs/ffi-support/0.4.2/ffi_support/struct.ExternError.html#layoutfields">https://docs.rs/ffi-support/0.4.2/ffi_support/struct.ExternError.html#layoutfields</a>, <a href="https://docs.rs/ffi-support/0.4.2/ffi_support/struct.ByteBuffer.html#layoutfields">https://docs.rs/ffi-support/0.4.2/ffi_support/struct.ByteBuffer.html#layoutfields</a></p>
<p>It was never clear to me how I'd change either of these if I was going to, since even a semver-major bump would likely be missed and cause silent failures though.</p>



<a name="218505916"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218505916" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218505916">(Dec 02 2020 at 01:39)</a>:</h4>
<p>Yeah, historically layout guarantees have been mostly a matter of documentation.</p>



<a name="218506004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506004">(Dec 02 2020 at 01:40)</a>:</h4>
<p><span class="user-mention" data-user-id="117495">@rpjohnst</span> I'm not arguing against an archetype-based approach. I'm just trying to document the pitfalls of <code>Muckable</code> in relation to the scope-aware approach. I'd love to explore an archetype-based approach again. The last attempt wasn't sound, though.</p>



<a name="218506016"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506016" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506016">(Dec 02 2020 at 01:40)</a>:</h4>
<p>An archetype system would presumably let you make semver-major changes actually break downstream code relying on the old layout, because their use of the safe transmute API would stop compiling, right?</p>



<a name="218506023"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506023" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506023">(Dec 02 2020 at 01:41)</a>:</h4>
<p>If you figure out how to do it soundly, presumably yes.</p>



<a name="218506049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506049">(Dec 02 2020 at 01:41)</a>:</h4>
<p>I think in this case no since the "transmutes" happen when passing/returning these over the FFI. I don't really see a solution to that, though.</p>



<a name="218506050"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506050" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506050">(Dec 02 2020 at 01:41)</a>:</h4>
<p>Sure- I didn't read you as arguing against archetypes. And I'm not arguing <em>for</em> <code>Muckable</code> either- I'm arguing against <code>Scope</code>.</p>



<a name="218506121"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506121" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506121">(Dec 02 2020 at 01:42)</a>:</h4>
<p>I can't really project how an archetype-based approach stacks up as a solution to this pitfall of <code>Muckable</code>, since we don't have a sound design of an archetype based approach.</p>



<a name="218506125"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506125" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506125">(Dec 02 2020 at 01:42)</a>:</h4>
<p><span class="user-mention" data-user-id="209168">@Thom Chiovoloni</span> Perhaps you could put some safer-transmute trait bounds around the FFI? I've not read anything beyond the comments you linked though.</p>



<a name="218506147"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506147" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506147">(Dec 02 2020 at 01:43)</a>:</h4>
<p>hm, perhaps. The danger is that the user fixes this but fails to update their matching C headers (or equivalent in other languages)</p>



<a name="218506149"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506149" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506149">(Dec 02 2020 at 01:43)</a>:</h4>
<p>Sure. My point is, a downside of <code>Muckable</code> isn't an upside of <code>Scope</code> per se, it's just an upside of "literally anything that's not <code>Muckable</code>." :P</p>



<a name="218506236"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506236" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506236">(Dec 02 2020 at 01:44)</a>:</h4>
<p>The <code>Muckable*</code> family of approaches is the only approach to stability we have right now that's sound. So, sure, this might be an upside to <em>other</em> approaches besides the foundational one, but those approaches don't really exist yet.</p>



<a name="218506379"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506379" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506379">(Dec 02 2020 at 01:46)</a>:</h4>
<p>This is kind of what I was getting at the other day talking about the problem space. We ought to figure out what kinds of properties we <em>want</em> to express (i.e. is partial visibility something we need, and if so which flavor of it) and then try to come up solution<em>s</em> that address that. Neither <code>Muckable</code> alone nor <code>Scope</code> really do that so listing downsides of <code>Muckable</code> seems... like a weird direction to take when we could be trying to make archetypes sound or coming up even other alternatives.</p>



<a name="218506433"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506433" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506433">(Dec 02 2020 at 01:47)</a>:</h4>
<p>Like, maybe the trait system simply isn't up to the task and we need to add new kinds of attributes or new things to label with <code>pub</code>.</p>



<a name="218506624"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506624" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506624">(Dec 02 2020 at 01:50)</a>:</h4>
<p>I'd like an API that can tell me whether a <em>given</em> transmutation is well-defined and safe.</p>



<a name="218506643"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506643" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506643">(Dec 02 2020 at 01:51)</a>:</h4>
<p>Anything that intertwines stability with safety fails to achieve that. <span aria-label="sad" class="emoji emoji-2639" role="img" title="sad">:sad:</span></p>



<a name="218506735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506735">(Dec 02 2020 at 01:52)</a>:</h4>
<p>I don't have any objection to stable APIs, and in fact I think the scope-aware approach is an unopinionated foundation for designing stable APIs.</p>



<a name="218506748"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506748" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506748">(Dec 02 2020 at 01:52)</a>:</h4>
<p>Frankly I still don't really follow that. The precise way that you're splitting up safety and stability doesn't seem to match how Rust works to me.</p>



<a name="218506842"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506842" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506842">(Dec 02 2020 at 01:54)</a>:</h4>
<ul>
<li><code>mem::align_of</code> tells me a type's alignment, without any semver stability guarantees.</li>
<li><code>mem::is_transmutable</code> tells me whether a given transmutation is well-defined and safe, without any stability guarantees.</li>
</ul>



<a name="218506864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506864">(Dec 02 2020 at 01:55)</a>:</h4>
<p>Well I think it's well-established at this point that <code>mem::align_of</code> (and <code>size_of</code> even moreso) are kind of weird and not a pattern we want to perpetuate.</p>



<a name="218506960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506960">(Dec 02 2020 at 01:56)</a>:</h4>
<p>I'm really surprised to hear that. These are really workhorse intrinsics, and I have a hard time imagining Rust without them.</p>



<a name="218506970"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506970" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506970">(Dec 02 2020 at 01:57)</a>:</h4>
<p>Those two things aren't contradictory. Unfortunately :P</p>



<a name="218506992"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218506992" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218506992">(Dec 02 2020 at 01:57)</a>:</h4>
<p>Has any design work been done on alternatives? If there's a model of how such layout-based intrinsics <em>ought</em> to be surfaced, I'd love to see it.</p>



<a name="218507058"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507058" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507058">(Dec 02 2020 at 01:58)</a>:</h4>
<p>The golden standard in my mind here is that, within a module, you can (without <code>unsafe</code>) do anything that's <em>valid</em> to the types you define. (So that includes transmutes that might break in future versions, but that's okay because they're in the same place as the breaking changes.)</p>
<p>And then outside, you can only do things that the type author has explicitly opted into as being part of its API. So you can't access private fields (they might change, or it might be unsafe, perhaps this is a conflation but I don't think that's relevant really), you can't construct or exhaustively-match a <code>#[non_exhaustive]</code> type, etc.</p>



<a name="218507073"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507073" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507073">(Dec 02 2020 at 01:58)</a>:</h4>
<p>I'm not aware of any. But if there were I would kind of expect it to come out of this project!</p>



<a name="218507118"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507118" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507118">(Dec 02 2020 at 02:00)</a>:</h4>
<p>So anyway the upshot of that golden standard is that, if you want to let consumers do transmutes that rely on some aspect of your type's layout, you ought to expose that layout via some kind of opt-in like a <code>pub</code> keyword or a trait impl.</p>



<a name="218507202"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507202" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507202">(Dec 02 2020 at 02:01)</a>:</h4>
<p>For example, if we were designing <code>mem::align_of</code> through this lens, I would expect there to be some kind of "public alignment" that consumers are allowed to rely on, and then the "actual alignment" that the type's module can exploit, which is perhaps greater than the public alignment.</p>



<a name="218507337"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507337" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507337">(Dec 02 2020 at 02:03)</a>:</h4>
<p>This seems like a worthy project, but <em>also</em> like a <em>really</em> significant undertaking — way beyond what would be a useful-but-intermediate foundation for safe transmutation.</p>



<a name="218507551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507551">(Dec 02 2020 at 02:06)</a>:</h4>
<p>A strength of the foundational API is that it's unopinionated. There's no stability system it rules out, because it <em>doesn't care about stability</em>. It's enough to make Safe <em>any</em> transmutation scenario this working group has considered, and it's unopinionated enough that it can serve as the foundation for any of the stability systems this working group has conceived of.</p>



<a name="218507580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507580">(Dec 02 2020 at 02:07)</a>:</h4>
<p>I don't think "it's a big job" is any kind of justification here. This is why I suggested a non-<code>Scope</code>-based magical intrinsic early on- <em>that's</em> an unopinionated useful way to expose "you can do any valid transmute inside a module, and only what the author opts into outside the module."</p>



<a name="218507669"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507669" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507669">(Dec 02 2020 at 02:09)</a>:</h4>
<p>Well, except that it doesn't cover a huge swath of transmutations the WG has considered. Namely: generic abstractions over transmutation. And this WG is <em>full</em> of people whose bread and butter is building safe generic abstractions over transmutations.</p>



<a name="218507739"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507739" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507739">(Dec 02 2020 at 02:10)</a>:</h4>
<p>That doesn't change the fact that exposing all valid transmutes outside of the module, even with "unstable" in the name, punches a big hole in the visibility and safety systems.</p>



<a name="218507781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507781">(Dec 02 2020 at 02:11)</a>:</h4>
<p>Whoa, what's the hole punched in visibility and safety? If you make a field visible to some set of scopes, you're saying that consumers in those scopes can fiddle with it arbitrarily. The API I'm proposing respects that.</p>



<a name="218507867"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507867" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507867">(Dec 02 2020 at 02:13)</a>:</h4>
<p>Well, you're right my point is a bit more nuanced than that. But consider what you keep saying- safe transmutation is totally new ground. So while you are respecting the set of scopes that have the right to read/write/match those fields, you are also giving those scopes a huge amount of extra information to exploit, while the current ecosystem simply does not expect changes to that information to be breaking.</p>



<a name="218507876"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507876" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507876">(Dec 02 2020 at 02:13)</a>:</h4>
<p>Which you of course understand, that's why you put <code>Unstable</code> in the name. :P</p>



<a name="218507949"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507949" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507949">(Dec 02 2020 at 02:14)</a>:</h4>
<p>So to be specific, the hole is "suddenly reordering fields is a breaking change (except by fiat we've declared that using <code>Unstable</code> traits contrary to their impls' documentation doesn't give you any guarantees so it's actually not a breaking change)."</p>



<a name="218507982"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218507982" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218507982">(Dec 02 2020 at 02:15)</a>:</h4>
<p>Ignoring the <code>Unstable</code> naming (because I don't agree that that's sufficient, <code>mem::size_of</code> et al notwithstanding), this is tantamount to retroactively declaring <code>#[repr(C)]</code> to be defining stable field order.</p>



<a name="218508030"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508030" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508030">(Dec 02 2020 at 02:16)</a>:</h4>
<p>No, reordering fields <em>isn't</em> a breaking change in a ton of contexts where people are abstracting over transmutations; e.g., <code>Zeroable</code>.</p>



<a name="218508053"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508053" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508053">(Dec 02 2020 at 02:17)</a>:</h4>
<p>Well, if you were only proposing <code>Zeroable</code> then that wouldn't be an issue. But the foundational <code>UnstableBla</code> API <em>does</em> let you write code that relies on field order, without the type author opting into field order stability.</p>



<a name="218508133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508133">(Dec 02 2020 at 02:19)</a>:</h4>
<p>I can <em>already</em> write safe code that relies on field order; it's not like field offsets are unobservable with safe code. Just as I can already write safe code that relies on the size, and the alignment.</p>



<a name="218508150"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508150" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508150">(Dec 02 2020 at 02:19)</a>:</h4>
<p>Can we at least start by trying to come to some kind of agreement about <code>mem::size_of</code>'s weirdness, then?</p>



<a name="218508207"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508207" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508207">(Dec 02 2020 at 02:20)</a>:</h4>
<p>I am not totally sure that you can cause downstream code to stop compiling simply by reordering fields, without relying on early-days APIs that didn't consider layout visibility.</p>



<a name="218508242"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508242" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508242">(Dec 02 2020 at 02:21)</a>:</h4>
<p>Like, even if you <em>can</em> it doesn't change my position! Being able to break downstream code by changing stuff you didn't mark as public/stable is a <em>bad thing</em> and just because it's possible due to earlier design oversights does not mean we should add <em>more</em> ways to do it.</p>



<a name="218508348"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508348" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Wrenn <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508348">(Dec 02 2020 at 02:23)</a>:</h4>
<p>I struggle to imagine a Rust without <code>size_of</code> and <code>align_of</code>. I <em>really</em> struggle to imagine a Rust where I can't cast a reference into a <code>usize</code>. This really just might be a fundamental difference in opinion between us.</p>



<a name="218508410"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/216762-project-safe-transmute/topic/Muckable%20Pitfall%20-%20Partial%20Visibility/near/218508410" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rpjohnst <a href="https://rust-lang.github.io/zulip_archive/stream/216762-project-safe-transmute/topic/Muckable.20Pitfall.20-.20Partial.20Visibility.html#218508410">(Dec 02 2020 at 02:24)</a>:</h4>
<p>I'm not asking you to imagine either of those things. I'm asking you to consider whether those things really are justification for adding more things like them.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>